00100		CHAPTER 9
00200	
00300			   MODEL EVALUATION
00400	9.1. Evaluation
00500		Evaluation procedures  for  models  involve  the  innocent-looking
00600	question - `how good is the model?' The ordinary language term
00700	`good' in general means praiseworthy. But what is a model  `good  as'
00800	or `good for' in order to be praiseworthy? A model can be deemed good
00900	as a representation or good for an application. Our  primary  aim  in
01000	constructing  this  model  was  to  explore  and test a theory having
01100	explanatory verisimilitude. To satisfy this aim the model  must  meet
01200	norms  of  internal  consistency and norms of external correspondence
01300	with observed phenomena. A secondary aim involved pragmatic norms  of
01400	application.  These  aims  are  not  unrelated  but the first is more
01500	fundamental  since  useful  applications  require  some   degree   of
01600	consistency and verisimilitude.
01700		A model in the form of an algorithm consists of a structure of
01800	mechanisms whose inner workings are sufficient to generate the outward
01900	behavior under consideration. The theory embodied in the model is
02000	revealed by the set of statements which describes how the structure
02100	reacts under various circumstances.
02200		Theories have many functions. They can be summarized as follows
02300	[from Bunge?]
02400		(1)To systematize knowledge.
02500		(2)To explain facts by showing how they are the entailed
02600	              consequences of the systematizing hypotheses.
02700		(3)To increase knowledge by deriving new facts.
02800		(4)To enhance the testability of hypotheses by connecting them 
02900	              to observations.
03000		(5)To guide research by:
03100	              (a)posing fruitful problems
03200	              (b)suggesting new data to gather
03300	              (c)opening new lines of investigation
03400		(6)To map a portion of reality.
03500		It is a tall order for a theory to fulfill all of these
03600	functions. In undeveloped fields we should be happy with even one  of
03700	them.   Models   can  be  assigned  these  functions  when  they  are
03800	theoretical, rather than replicative, in type. Our model was intended
03900	primarily to serve functions (2) and (4), testable explanation.
04000		What constitutes a satisfactory explanation has been  treated
04100	in  section  00.0.  The  `fit'  or  correspondence  with phenomena as
04200	indicated by measurements and empirical tests indicte truth,or grains
04300	of  truth showing  promise for turning out to be accepted as true. Our tests and
04400	measures were described in section 000.0. 
04410		Decision procedures for a consensus  acceptability  of a model
04500	depend not so much on truthlikehood, an elusive state, as
04600	on whether a majority of the relevant expert community  believes  the
04700	theory  or  model to approximate truth to some unknown and unknowable
04800	degree and be better than their rivals. Truth or  falsity  cannot  be
04900	proven  with certainty but their presence can be assayed by some sort
05000	of critical assesment and deliberation. A theory or model  may  bring
05100	cognitive or pragmatic comfort, not because it is TRUE but because
05200	it represents an improvement  over its  contending  rivals.  Cognitive
05300	comfort  is  a  type  of  intellectual  satisfaction  while pragmatic
05400	comfort accrues from applications to problems in order to make things
05500	work the way humans want them to work optimally in practical contexts 
05600	of action.
05700	NEW EVIDENTIAL SUPPORT
05800		It would  be  a  bonus  if  our  model  could  satisfy  those
05900	interested   in  function  (3)  listed  above,  making  possible  new
06000	knowledge  through  prediction.  It   would   give   clinicians   and
06100	investigators  something to look for. This novelty could arise in two
06200	ways. First the model might demonstrate a property  of  the  paranoid
06300	mode  hitherto  unobserved clinically.   In principle this could come
06400	about because the I/O behavior of the model is  a  consequence  of  a
06500	large   number  of  interacting  hypotheses  and  assumptions  chosen
06600	initially chosen to explain frequently observed phenomena.  When  the
06700	elements  of  such  a  complex  conjunction  interact with input they
06800	generate consequences in addition to  those  they  were  designed  to
06900	explain.  Whether  any  of  these  consequences  are  significant  or
07000	characteristic of the paranoid mode  remains  a  subject  for  future
07100	study.
07200		A second source of novelty would lie in the behavior of the model
07300	in some new situation. Since it is designed to simulate communicative
07400	behavior in an interview situation, the `new' circumstance would have to
07500	involve some new type of linguistic interaction which the model is capable
07600	of responding to. From its behavior one might then predict how paranoid
07700	patients would behave under similar circumstances. The requisite
07800	empirical tests and measures would show the degree of correspondence
07900	between patient and model behaviors.
08000		This possibility is of importance  in  considering  therapies
08100	for  patients  tangled in the quandaries of the paranoid mode.  Since
08200	the  model  operates  at  a  symbol processing  level  using  natural
08300	language, it is at this level at which linguistic and semantic skills
08400	of clinicians can be applied. Language-based or semantic  techniques  do  not  seem
08500	very effective in the psychoses
08600	but they are useful in states of lesser severity. A wide range of
08700	new semantic techniques, including extremes, could be tried first on
08800	the model without hurting patients through blind experimentation.
08900		While we have used the model to explore a theory and to
09000	study psychiatric judgements, its potential use as a training device
09100	has not escaped our attention. Medical students and psychiatric residents
09200	need `disposable' patients to practice on without fear of harming the
09300	patient. The paranoid model can print out a trace of its inner states 
09400	during and after an interview. Whether the optimal goal of interviewing
09500	gathering relevant information without upsetting the patient, has been
09600	achieved, thus can be estimated. A beginning interviewer can practice
09700	in private or with a supervisor present. Many interviewers have reported
09800	that the model has a definite effect on them. The student can get the
09900	feel of the paranoid mode long before he interviews an actual patient.
10000	The effect of various interviewing styles can be studied and compared.
10100	We are currently using the model to train para-psychiatric personnel  
10200	to identify and evaluated paranoid linguistic communication.
10300	9.3. Extra-evidential support 
10400		Besides the function of evidence in evaluating models, there is
10500	is the role of extra-evidential support. This support derives from
10600	plausibility arguments, comparisons with competing models or theories,
10700	and coherence with other domain theories.
10800		A theoretical model is evaluated relative to rival explanations.
10900	Our model stands as a candidate for a consensus ethogenic explanation
11000	of paranoid processes. The expert forum will decide its ultimate status.
11100	A theoretical model is partial, perspectival and has a short half-life.
11200	Hopefully it lives long enough to provide a first approximation from
11300	which better approximations can develop.
11400	(More here on plausibility, competetors, and other domain theories)
11500	(more comprehensive= covers more facts→ greater explanatory power)